Bayes' Theorem

Given events \(A\) and \(B\) in the same sample space \(P(B) \neq 0\)

\[ P(A \mid B) = \frac{P(A)P(B \mid A)}{P(B)}\]

The main utility of Bayes' theorem is in reversing the order of dependence of events. In many circumstances, two events may be dependent on each other because one causes another (in the probabilistic sense), but it is useful to know given the effect, what is the probability that the cause is also present.

Proof

Bayes' theorem is very simple consequence of the multiplicative rule of probability, which itself simply follows from the definition of conditional probability. That is, we have that:

\[ P(A \mid B) = \frac{P(A \cap B)}{P(B)} \quad \text{and} \quad P(B \mid A) = \frac{P(A \cap B)}{P(A)}\]

and hence by equating \(P(A \cap B)\) we know that:

\[ P(A \mid B)P(B) = P(B \mid A)P(A).\]

We then divide by \(P(B)\) to get the desired result:

\[P(A \mid B) = \frac{P(A)P(B \mid A)}{P(B)}\]